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Abstract 

Several authors have examined connections between permutations which avoid 132, 
continued fractions, and Chebyshev polynomials of the second kind. In this paper we 
prove analogues of some of these results for permutations which avoid 1243 and 2143. 
Using tools developed to prove these analogues, we give enumerations and generating 
functions for permutations which avoid 1243, 2143, and certain additional patterns. We 
also give generating functions for permutations which avoid 1243 and 2143 and con- 
tain certain additional patterns exactly once. In all cases we express these generating 
functions in terms of Chebyshev polynomials of the second kind. 

Keywords: Restricted permutation; pattern-avoiding permutation; forbidden sub- 
sequence; continued fraction; Chebyshev polynomial 



1 Introduction and Notation 

Let & n denote the set of permutations of {1, . . . , n}, written in one-line notation, and suppose 
7r G & n and a G &k- We say a subsequence of n has type a whenever it has all of the 
same pairwise comparisons as a. For example, the subsequence 2869 of the permutation 
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214538769 has type 1324. We say ir avoids a whenever re contains no subsequence of type 
a. For example, the permutation 214538769 avoids 312 and 2413, but it has 2586 as a 
subsequence so it does not avoid 1243. If 7r avoids a then a is sometimes called a pattern 
or a forbidden subsequence and n is sometimes called a restricted permutation or a pattern- 
avoiding permutation. In this paper we will be interested in permutations which avoid several 
patterns, so for any set R of permutations we write & n (R) to denote the elements of & n which 
avoid every pattern in R and we write &(R) to denote the set of all permutations (including 
the empty permutation) which avoid every pattern in R. When R = {ttj,^, • • • , 7r r } we 
often write & n (R) = &n(^i,^2, ■ ■ ■ ,^ r ) and &(R) = ©(7Ti,7r 2 , . . . , 7r r ). 

Several authors f|, || §, [12|, [L6| have shown that generating functions for (5(132) with 
respect to the number of subsequences of type 12 ... k, for various collections of values of k, 
can be expressed as continued fractions. The most general result along these lines, which 
appears as ||, Theorem 1], states that 
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Here ^(tt) is the number of subsequences of type 12 . . . k in tt. Generating functions for 
6(132) have also been found to be expressible in terms of Chebyshev polynomials of the 
second kind |5|, |9], [12|, |13| . One result along these lines, which appears as || Theorem 2] , |12| , 
Theorem 3.1], and || Theorem 3.6, second case], states that 
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Here U n (x) is the nth Chebyshev polynomial of the second kind, which may be defined by 
(n + 

sint 



sin((n + l)t) 

U n (cost) = : . Another result along these lines, which appears as R Theorem 



3], states that 
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E 1 " 1 = Eri C- V' - l ) {u r^lL (3) 



Here the sum on the left is over all permutations in (3(132) which contain exactly r subse- 
quences of type 12 ... k, the quantity \tt\ is the length of n, and the sum on the right is over 
all sequences l±, ■ ■ ■ , h of nonnegative integers such that Y^i^i^V-i) = r - ^ or °ther 



results involving (5(132) and continued fractions or Chebyshev polynomials, see [15 and the 
references therein. 

Permutations which avoid 1243 and 2143 are known to have many properties which are 
analogous to properties of permutations which avoid 132. For instance, it is well known that 
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|(3 n (132)| = C n for all n > 0, where C n is the nth Catalan number, which may be defined 
by C = 1 and 

n 

C n = J2^-iC n . t (n>l). 

i=i 

(The Catalan number C n may also be defined by C n = ;^i( 2 ™)-) As a result, for all n > 0, 
the set 6 n (132) is in bijection with the set of Catalan paths. These are the lattice paths 
from (0,0) to (n, n) which contain only east (1,0) and north (0, 1) steps and which do not 
pass below the line y — x. Kremer [|lfj, Corollary 9] has shown that |(5 n (1243, 2143) | = r n _i 
for all n > 1, where r n is the nth Schroder number, which may be defined by r$ = 1 and 

n 

r n = r n _i + Erj_ir n _j (n > 1). 
i=i 

As a result, for all n > 0, the set 6„, + i(1243, 2143) is in bijection with the set S n of Schroder 
paths. These are the lattice paths from (0,0) to (n,n) which contain only east (1,0), north 
(0,1), and diagonal (1,1) steps and which do not pass below the line y = x. We write 
S to denote the set of all Schroder paths (including the empty path). In view of this 
relationship, we refer to permutations which avoid 1243 and 2143 as Schroder permutations. 
(For more information on pattern-avoiding permutations counted by the Schroder numbers, 
see p, [7], [10], |19| . For generalizations of some of these results, see 0, [TTJ. For a partial list of 



other combinatorial objects counted by the Schroder numbers, see flTj| pp. 239-240].) 

Motivated by the parallels between (3(132) and (5(1243,2143), in this paper we prove 
analogues of (|IJ), (||), (H), and several similar results for (5(1243,2143). We begin with 
some results concerning 6(1243,2143) and continued fractions. We first define statistics Tk, 
k > 1, on S and 6(1243,2143). On 6(1243,2143), the statistic r k is simply the number of 
subsequences of type 12 . . . k. On S, the statistic Tk is a sum of binomial coefficients over 
east and diagonal steps. We then give a combinatorial definition of a bijection (p : S —>■ 
6(1243, 2143) with the property that Tk((p(ir)) = Tkiji) for all k > 1 and all % G S. Using ip 
and a result of Flajolet 0, Theorem 1], we prove the following analogue of ([!]). 

e n x i k(w) = 1 + — Xl XIX2 — • (4) 

?re 6(1243,2143) fc>l I — X\ 2 

1 — X\X2 — 2 

1 — X1X2X3 — ' ' ' 

By specializing the x^s in (|4]), we obtain continued fraction expansions for several other 
statistics on 6(1243,2143). In particular, we show that for all k > 1, 

oo 

El©n(1243,2143, 12...k)\x n = 1 + — m . 

n=0 1 >E 'rj- 



1 — X • • • 

Here the continued fraction on the right has k — 1 denominators. Following 01], we then 
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define a Schroder continued fraction to be a continued fraction of the form 



1 + ^ 

1 - m =^ 

1 - mi fn 3 

1 - m 2 - 

1 - m 3 

where rrti is a finite monic monomial in a given set of variables for alH > 0. We prove that 
the multivariate generating function for a countable family of statistics on ©(1243,2143) 
can be expressed as a Schroder continued fraction if and only if each statistic is a (possibly 
infinite) linear combination of the r^s and each appears in only finitely many of these 
linear combinations. This result is an analogue of j|, Theorem 2]. 

We then turn our attention to analogues of @) and (|3]). For any k > 2 and any a G &k-i 
we give the generating function for |@ n (1243, 2143, ka)\ in terms of the generating function 
for |©„(1243, 2143, a)\. Using this result, we show that 

J^|(5 n (i243,2i43, 12...k)\x n = 1 + 

n=0 U k -i 55§ 



and 



00 \/xU k ^- 2 i 2^ 

^|6„(1243,2143,213...A;)|x n = 1 + 7^~\~' 

n=0 t/" fc _i (^J 

for all k > 1. Both of these results are analogues of @. We then use </2 and some well-known 
results concerning lattice paths to show that 
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Here the sum on the left is over all permutations in (5(1243,2143) which contain exactly r 
subsequences of type 12 ... k and the sum on the right is over all sequences . . . , lb, and 
mo, nii, ... ,rrib of nonnegative integers such that r = ^i=o(^ + m «) (^fc-^ 1 ) ■ ^his resu lt is an 
analogue of (|3|). 

In the next two sections of the paper we give enumerations and generating functions for 
various sets of permutations in (3(1243,2143). For instance, we show that 

V|6 n (1243, 2143, 2134... k)\x n = 1 + x ^=^$ (5) 
^ fk{x) 



n=0 

and 



V |e n (1243,2143,3214...A;)|x n = 1 + x ^^$ (6) 
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for all k > 3. Here f 2 (x) = (x - l) 2 , 



h(x) = (1 - 2x) ! ( v 'i) t - 3 t4-3 f Vt^I - (1 - xf ' ' ~ ' 



for all k > 3, and 



<fc(z) = -(1 + 2x - x 2 )(v^) fc+2 f/ fc (V/f ) + - 4a;3 + 2 ^ + (V/f ) 

for all k > 2. Setting = 3 in (|5|) and (0), we find that 

|6 n (1243, 2143, 231)| = (n + 2)2"- 3 (n > 2) (7) 



and 



|6„(1243,2143,321)|= ( n Q M + P x 1 )+2( B 2 1 )+ 2 r 3 1 ) ^ ^ ( 8 ) 



It is an open problem to provide combinatorial proofs of (0) and (H). We also show that 

5>M 



_ x(l + x)(l — x) 2 
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where the sum on the left is over all permutations in (3(1243,2143) which contain exactly 
one subsequence of type 213 . . . k. It is an open problem to give the sum x ^ m closed 

form when it is over all permutations in ©(1243, 2143) which contain exactly r subsequences 
of type 213 ... A;, where r > 2. 

We conclude the paper by collecting several open problems related to this work. 



2 Statistics and a Product for Schroder Paths 

In this section we define a family of statistics on Schroder paths. We then recall the first- 
return product on Schroder paths and describe the behavior of our statistics with respect 
to this product. We begin by recalling the height of an east or diagonal step in a Schroder 
path. 

Definition 2.1 Let n denote a Schroder path, let s denote a step in 7r which is either east 
or diagonal, and let (x,y) denote the coordinates of the left-most point in it. We define the 
height of s, written ht(s), by setting ht(s) = y — x. 

We now define our family of statistics on Schroder paths. 
Definition 2.2 For any Schroder path ir and any positive integer k we write 

where the sum on the right is over all east and diagonal steps in tt. Here we take Q) = 
whenever j < or i < j . For notational convenience we set tq(tt) = for any Schroder path 

TT. 
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Example 2.3 Let it denote the Schroder path given in Figure 1, so that n is given by n = 
NDEDNNNNDNEENEDEEE. Then t^tt) = 12, t 2 (tt) = 28, t 3 (tt) = 35, t 4 (tt) = 24, 
75(71") = 8, tq(ty) = I, and t^tt) = for all k > 7. 
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Figure 1: The Schroder path of Example |2.3j . 

Before we recall the first-return product for Schroder paths, we make an observation 
regarding those paths in S n which begin with a diagonal step. 
Proposition 2.4 (i) For all n > 1, the map 

<Sn-i — ► <Sn 

71 I — > D, IT 

is a bijection between 5 n _i and the set of Schroder paths in S n which begin with a 
diagonal step. 

(ii) Ti(D, 7r) = 1 + Ti(n) for all tx eS. 

(iii) T^D, 7r) = Tfc(7r) for all k > 2 and all tt G S. 

Proof, (i) This is immediate. 

(ii),(iii) From (|9|) we find that for all k > 1, 



T fc (D,7r) = 




Now (ii) and (iii) follow. □ 

We now define the first-return product on Schroder paths. 
Definition 2.5 For any Schroder paths tx\ and 712 we write 

7Ti * 7T 2 = N7TiE7T2. 
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Proposition 2.6 Let i and n denote positive integers such that 1 < i < n. Then the 
following hold. 

(i) The map 

<Sj_i x iS n _j > S n 

(7Ti,7r 2 ) H-> 7Ti * 7T 2 

zs a bisection between S%-\ x <S n _j and i/ie set 0/ Schroder paths in S n which begin with 
a north step and first touch the line y = x at (i, i). 

(ii) For all k > 1, all tt\ G <Si_i, and all tt 2 G <Sn-i we /iai>e 

^(tti * 7r 2 ) = T k [ni) + r fc _i(7Ti) + T k (n 2 ). (10) 

Proof, (i) This is immediate. 

(ii) Fix A; > 1. Using @ we have 

V 7 V 7 Sg7Tl V 7 V 7 V 7 Sg7T 2 V 7 

= r fc (7Ti) + r fe _i(7Ti) + r fe (7r 2 ), 
as desired. □ 

3 Statistics and a Product for Schroder Permutations 

In this section we define a natural family of statistics on Schroder permutations which is 
analogous to the family of statistics we have defined on Schroder paths. We then describe 
a "product" on Schroder permutations which behaves nicely with respect to our statistics. 
This product is analogous to the first-return product for Schroder paths given in the previous 
section. We begin with our family of statistics. 

Definition 3.1 For any positive integer k and any permutation tt, we write t^Itc) to denote 
the number of increasing subsequences of length k which are contained in tt. For notational 
convenience we set r (7r) = for any permutation it. 

Observe that for any permutation tt, the quantity ti(tt) is the length of tt; we sometimes 
write \tt\ to denote this quantity. 

Example 3.2 Ifn = 71824356 thenr^Tx) = 8, t 2 (tt) = 16, t 3 (tt) = 16, t 4 (tt) = 9, t 5 (tt) = 2, 
and 7fc(7r) = for all k > 6. 

Observe that we have now defined Tfc(7r) when it is a Schroder permutation and when tt 
is a Schroder path. This will not cause confusion, however, since it will always be clear from 
the context which definition is intended. 

Before we describe our product for Schroder permutations, we make an observation re- 
garding those Schroder permutations whose largest element appears first. 
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Proposition 3.3 (i) For all n> 1, the map 

e„_i(1243,2143) — ► 6„(1243,2143) 

7T I— > Tl, TT 

is a bijection between 6 n _i(1243, 2143) and the set of permutations in (5 n (1243, 2143) 
which begin with n. 

(ii) 7i(n, tt) = 1 + Ti(tt) /or all n > 1 and all tt G 6 n _i(1243, 2143). 

(iii) Tfc(n,7r) = t^tt) for all k>2, all n> 1, and all n G (5 n _i(1243, 2143). 

Proof. (i) It is clear that the given map is one-to-one and that if n, n is a permuta- 
tion in 6„(1243,2143) then tt G 6„_i(1243, 2143), so it is sufficient to show that if tt G 
e n _i(1243, 2143) then n, tt avoids 1243 and 2143. To this end, suppose n G 6 n _i (1243, 2143). 
Since n avoids 1243 and 2143, in any pattern of either type in n, 7r the n must play the role 
of the 4. But this is impossible, since the n is the left-most element of n, tt, but 4 is not the 
left-most element of 1243 or 2143. Therefore n, n avoids 1243 and 2143. 

(ii) This is immediate, since Ti(tt) is the length of n for any permutation tt. 

(iii) Since n is both the largest and the left-most element in n, tt, it cannot participate in 
an increasing subsequence of length two or more. Therefore any such subsequence in n, n is 
contained in n, and (iii) follows. □ 

We now describe our product for Schroder permutations. To do so, we first set some 
notation. 

Let 7Ti and 7i2 denote nonempty Schroder permutations. We write #i to denote the 
sequence obtained by adding j^l — 1 to every entry in tt\ and then replacing j^l (the 
smallest entry in the resulting sequence) with the left-most entry of 7r 2 . We observe that #i 
has type 7Ti. We write if 2 to denote the sequence obtained from 7r 2 by removing its left-most 
element . 

Definition 3.4 For any nonempty Schroder permutations tti and 7r 2; we write 

Til * 7T 2 = 7fin7f 2 , 

where n = | ttx | + K2I an d #1 and 7f 2 are the sequences described in the previous paragraph. 

Example 3.5 // 7Ti = 3124 and n 2 = 15342 then 7fi = 7168, n 2 = 5342, and 7Ti * tt 2 = 
716895342. 

Proposition 3.6 Let i and n denote positive integers such that 1 < i < n — 1. Then the 
following hold. 

(i) The map 

6^(1243,2143) x 6„_ i (1243, 2143) — ► 6„(1243, 2143) 

(7Ti,7T 2 ) H-> 7Ti*7r 2 

is a bijection between 6^(1243,2143) x 6 n ^.j(1243, 2143) and the set of permutations 
in 6 n (1243, 2143) for which tt(z + 1) = n. 
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(ii) For all k>\, all m E 6;(1243, 2143), and all vr 2 E 6^(1243, 2143) we have 



7fc(7Ti * 7T 2 ) = T k {-Kx) + Tfc-ifal) + T k (n 2 ) . (11) 

Proo/. (i) It is routine to verify that if m E 6 4 (1243, 2143) and vr 2 E 6 n _j(1243, 2143) then 
7Ti * 7r 2 E (5 n (1243, 2143), so it is sufficient to show that the given map is a bijection. We do 
this by describing its inverse. 

Suppose 7r E <5„(1243, 2143) and 7r(z + 1) = n. Let /i(7r) denote the type of the subse- 
quence 7r(l), 7t(2), . . . , 7r(i) of 7r and let / 2 (7r) denote the permutation of 1, 2, . . . , n — i which 
appears in n. Since 7r E 6 n (1243, 2143) and n contains subsequences of type /i(7r) and / 2 (7r) 
we find that /i(tt) E 6^(1243,2143) and / 2 (tt) G 6 n _;(1243, 2143). We now show that the 
map 7r i — > (fi(it), C 71 ") ) is the inverse of the map (7i"i, 7r 2 ) i— > 7Tj * 7r 2 . 

It is clear from the construction of 7Tl * 7r 2 that /i(7Ti * 7r 2 ) = 7Ti and / 2 (7Ti * ir 2 ) = n 2 , 
so it remains to show that /i(7r) * / 2 (?r) = 7T. To this end, observe that since 7r avoids 1243 
and 2143 and has 7r(i + 1) = n, exactly one of 1,2, ...,n — i appears to the left of n in 
7T. (If two or more of 1, 2, . . . ,ra — i appeared to the left of n then two of these elements, 
together with n and some element to the right of n, would form a subsequence of type 
1243 or 2143.) Therefore, the remaining elements among 1,2, . . . ,n — % are exactly those 
elements which appear to the right of n. It follows that /i(vr) * f 2 (ir) = it. Therefore the map 
7T i — ► (/i(tt), / 2 (7r)) is the inverse of the map (7ri,7r 2 ) h- > tti * 7r 2 , so the latter is a bijection, 
as desired. 

(ii) Let if i be as in the paragraph above Definition |3.4| . Observe that since #1 consists of 
exactly those elements of 7Ti*7t 2 which are to the left of n, there is a one-to-one correspondence 
between increasing subsequences of length k — 1 in ifi and increasing subsequences of length 
k in 7i"i * 7r 2 which involve n. Since 7fi and n\ have the same type, there are Tfe_i(7Ti) of these 
subsequences. Now observe that if an increasing subsequence of length k in 7ii * n 2 does not 
involve n, and involves an element of ifi other than the smallest element, then it is entirely 
contained in ifi. Similarly, observe that if an increasing subsequence of length k in tt\ * tt 2 
involves an element of n 2 other than the left-most element, then it is entirely contained in 
7r 2 . Therefore every increasing subsequence of length k in 7Ti * rc 2 which does not involve n 
is an increasing subsequence of length k in ifi or in n 2 . Since 7F1 and 7Tx have the same type, 
there are 77.(711) + r fc (7r 2 ) of these subsequences. Now (ii) follows. □ 

Although the results we have given in this section are sufficient for our current purposes, 
we remark that there are more general results along the same lines. For example, following 
P| and [|TTJ, let T k {k > 3) denote the set of all permutations in & k which end with k,k — l. 



Observe that T 3 = {132} and T 4 = {1243,2143}. Then there are natural analogues of all of 
the results in this section for &(T k ), where k > 5. 



4 A Bijection Between S n and & n+l (1243, 2143) 

Comparing Propositions |3.3| and |3.6| with Propositions |2.4j and |2.6| respectively, we see that 
for all n > there exists a bijection tp : S n — > (5 n+ i(1243, 2143) such that T k {ir) = T k ((p(7r)) 
for all 7r E S n . So far, we have only seen how to compute this bijection recursively. In this 
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section we use techniques of Bandlow and Killpatrick f2| and Bandlow, Egge, and Killpatrick 
U to compute this bijection directly. 

To define our bijection, we first need to introduce some notation. For all n > and all 
i such that 1 < i < n — 1, we write Sj to denote the map from & n to & n which acts by 
interchanging the elements in positions i and i + 1 of the given permutation. For example, 
si (354126) = 534126 and s 4 (354126) = 354216. We apply these maps from right to left, so 
that SiSj^Ti) = Si(sj(ii)). 

Suppose 7r G «S n . We now describe how to construct the image (p{tt) of n under our 
bijection ip. To illustrate the procedure, we give a running example in which the Schroder 
path Ti is given by tt = NDNNEEENNDENEE, which is illustrated in Figure 2 below. 



Figure 2: The Schroder path 7r used in the running example. 

To begin, label each upper triangle (i.e. each triangle whose vertices have coordinates of 
the form {i — 1, j — 1), (i — 1, j), (i, j)) which is below tt and above the line y = x with an Sj, 
where i is the x-coordinate of the upper right corner of the triangle. In Figure 3 below we have 
labeled the appropriate triangles for tt with the subscripts for the SjS. Now let a\ denote the 
sequence of s^s which begins with the s« furthest up and to the right and extends diagonally 
to the lower left, immediately above the line y = x, proceeding until it reaches a north step. 
Let 02 denote the sequence of s^s which begins immediately to the left of the beginning of a\ 
and extends diagonally to the lower left, immediately above <7i, until it reaches a north step. 
Construct 03, 04, ... in this fashion until all of the s^s above and to the left of o\ have been 
read. Repeat this process with the part of the path below the last east step before the north 
step which ended a x . In this way we obtain a sequence a%, . . . , G\. of maps on & n . In Figure 
4 below we have indicated the sequences 01, 03, 04, and a 5 for our example path tt. In this 
example we have o\ = ssS7Ses^, 02 = S7, 03 = sqs$, 04 = S4S3S2S1, 05 = S3S2, and 06 = £2- 

We this set-up in mind, we can now define y?(7r). 

Definition 4.1 For any Schroder path ti, let o~i, 02, . . . , 0fc denote the maps determined as 
in the discussion above. Then we write 

(p(n) = 0fc0fc_i . . . 01 (n + 1, n,n - 1, ... ,3,2, 1). 

Summarizing our running example, we have the following. 
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Figure 3: The Schroder path tt with triangles labeled. 



Example 4.2 Let n denote the Schroder path in Figure 2 above, so that n is given by 
7r = NDNNEEENNDENEE. Then o\ = ssSjSqs 5 , a 2 = s 7 , 03 = s e s 5 , o\ = S4S3S2S1, 05 = 
s 3 s 2,c"6 = s 2, and (p(ir) = 836791425. 

Example 4.3 Let tt denote the Schroder path given by n = DNEDDNNENDEE . Then 
o\ = s 8 s 7 s 6 s Fj , a 2 = S 7 S 6, °~3 — s 5; °~4 = s 2, an d V 9 ! 71 ") = 978624135. 

Example 4.4 Let tt denote the Schroder path given by ir = NNENDNNEDEEDE. Then 
0i = S8S7S6S5S4S3S2S1, o~ 2 = s 6 s 5 S4S 3 s 2 , cr 3 = S5S4S3, o- A = s 3 , <7 5 = Si, and (p(n) = 
683425719. 

Our goal for the remainder of this section is to show that the map ip is a bijection which 
preserves the statistics r k for all k > 1. To do this, we first consider the value of <p on a 
Schroder path which begins with a diagonal step. 

Proposition 4.5 For all n > and all ir E S n we have 

<p(D,n) = n + l,tp(n). (12) 

Proof. Observe that by the construction of (p(D,n) no Si will appear in a k a k _i . . . a x . 
Therefore the left-most element of n + 1, n, n — 1, . . . , 2, 1 will not be moved by o k o k ^\ . . . o\. 
Now the result follows. □ 

Next we consider the value of ip on a product of two Schroder paths. 
Proposition 4.6 For all Schroder paths tx\ and 7r 2 we have 

ip(7Ti * tv 2 ) = (p(ni) * p(n 2 ). (13) 

Proof. In the construction of ipfai * n n ), let Oi, . . . , o~j denote the strings of s^s constructed 
from triangles below 7r 2 , and observe that the next string constructed will be SjSj_i . . . si, 
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Figure 4: The Schroder path tr with ai, a 3 , cr 4 , and o~ 5 indicated. 



where 7Ti ends at (i — 1, i — 1). Now let 0^+2, . . . cr^ denote the remaining strings and let a 
denote the left-most element of ^(7^). Then we have 

cr(7Ti*7r 2 ) = a k ...o-j + 2S i Si-i...s 1 aj...ai(n + l,n,...,2,l) 

= a k . ..a j+2 SiSi^i . . . si(n + l,n,...,n-i+ l,a,<pfa)) 
= a k . . . aj + 2(n, n — 1, . . . , n — % + 1, a, n + 1, ^(7^)) 
= y?(7Ti),n + l,^(vr 2 ) 
= (pfa) * <p(ir 2 ), 

as desired. □ 

We now show that ip is a bijection from «S n to © n+ i(1243, 2143). 
Proposition 4.7 (i) For a// rr e S, we have ip(n) E 6 n+ i(1243, 2143). 

(ii) For all n > the map ip : S n — > 6 n+ i(1243, 2143) is a bijection. 

Proof, (i) Observe that y?(0) = 1, <p(D) = 21, and p(NE) = 12, so (i) holds for all n e S 
and all 7T G <Si. Arguing by induction, suppose (i) holds for all 7r G «Sj, where < i < n, 
and fix 7r G <S„. If tt begins with a diagonal step then 7r = D, 7Tj and by ( |i~2|) we have 
99(7r) = n + l,(^(7Ti) G 6 n+ i(1243, 2143). If tc does not begin with a diagonal step then by 
Proposition |2.6| (i) there exist iii G <Sj_i and 7r2 G <S„_i, where 1 < z < n, such that 7r = 7Ti*7T2. 
Then by (JT|) we have ip(ir) = (pfa) * ipfa) G 6 n+ i(1243, 2143). Now (i) follows. 

(ii) First observe that by (i) and since |«S n | = |6 n+ i(1243, 2143) | for all n > 0, it is 
sufficient to show that (p is surjective. To do this, we argue by induction on n. 
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It is routine to verify that (f is surjective for n = and n = 1, so suppose by induction 
that ip is surjective for 1, 2, . . . , n — 1 and fix 7r G (5 n+ i(1243, 2143). If 7r(l) = n + 1 then by 
Proposition |3.3| (i) there exists n\ G © ra (1243, 2143) such that re = n + 1,tti. By induction 
there exists «i G 5 n -i such that <^(ai) = tti. We now have 



<p(D,an) = n + l,7n (by(0 



7T. 



If 7r(l) 7^ n + 1 then there exists i, 1 < i < n — 1, such that n(i + 1) = n + l. By Proposition 
U(i) there exist tti G 6,(1243,2143) and tt 2 G 6 n _ m (1243, 2143) such that tt = tti * tt 2 . 
By induction there exist a.\ G <Sj_i and a 2 G iS n _j such that ^(ai) = 7i"i and 99(02) = vr 2 . We 
now have 

(p(ax * a 2 ) = 7ri * tt 2 (by (P|)) 



= 7T. 

It follows that is surjective, as desired. □ 

We now show that p preserves the statistics r k for all k > 1. 
Proposition 4.8 For any k > 1 and any Schroder path tc we have 

T k {(p(ir)) = r k (n). (14) 

Proof. Suppose ir G S n ; we argue by induction on n. The cases n = and n = 1 are routine 
to verify, so suppose the result holds for all 7r G Si, where i < n — 1, and fix 7r G <S n . If tt 
begins with a diagonal step then 7r = Z), 7Ti for some tci G iS n -i and we have 

T k ((p(ir)) = r fc (^(D,7Ti)) 

= nin + l^im)) (by©) 

= tfjfci + 7jfe(yj(7ri)) (by Proposition 0(ii),(iii)) 



= + Tjfe(7Ti) (by induction) 

= r k (D, 7Ti) (by Proposition U(ii),(iii)) 

= T- fc (7r). 

If 7r does not begin with a diagonal step then 7r = 7i"i * 7r 2 for some 7i"i G «Si_i and 7r 2 G S n -i 
and we have 



T k ((p(n)) = T k ((p(7Ti * 7r 2 )) (by Proposition ^6|(i)) 

= rfc^Tn)*^)) (by©) 

= 7ife(^(7ri)) + r fe _i(v?(7n)) + r k (ip(n 2 )) (by (0)) 

= Tfc(7Ti) + r fe _i(7Ti) + r fe (7r 2 ) (by induction) 

= 7ifc(7ri*7r 2 ) (by(0)) 
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as desired. □ 



In view of Proposition |4l], the map ip also relates the Schroder paths of height at most 
k — 2 with Schroder permutations which avoid 12 ... k. In particular, we have the following 
result. 

Corollary 4.9 Fix k > 2 and let S nj k denote the set of Schroder paths in S n which do not 
cross the line y — x = k — 2. Then the restriction of (p to S n ^ is a bisection between S n ^ and 
© n+1 (1243,2143,12...Jfe). 

Proof. Observe that tt G 6 n+ i(1243, 2143) avoids 12 ... Jfe if and only if r k (ix) =0. By (0) 
this occurs if and only if Tfc(<^ _1 (7r)) = 0, which occurs if and only if c/9 _1 (7r) does not cross 
the line y — x = k — 2. □ 

For all n > 0, let T> n denote the set of all Schroder paths from (0,0) to (n,n) which 
contain no diagonal steps. (Such paths are sometimes called Catalan paths, because \V n \ is 
the well-known Catalan number C n = ^zi( ™) f° r & U n ^ 0.) In || Krattenthaler gives a 
bijection <f> : (5 n (132) — > V n such that 

rfc(vr) = T k (<j>(ir)) 

for all k > 1, all n > 0, and all n G (5 n (132). Krattenthaler's bijection <f> is closely related 
to our bijection (p. For any permutation tt, let ir denote the sequence obtained by adding 
one to every entry in tt. Observe that the map uj : S n (132) — > & n +i (1243, 2143) given by 
w{tt) — l,7r for all tt G S n (132) is a bijection between (3 n (132) and the set of permutations 
in (5 n+ i(1243, 2143) which begin with 1. The bijections 4> and tp are related in that 

0(tt) = ip- 1 ^^)) 

for all n > and all vr G 6„(132). 

5 The Continued Fractions 

In this section we will encounter several continued fractions, for which we will use the fol- 
lowing notation. 

Definition 5.1 For any given expressions a» (i > 0) and hi (i > 0) we write 

Oq Oi 02 03 

b + h + b 2 + 6 3 + 
to denote the infinite continued fraction 

a 



ai 

b + 



h H 



a 3 



a 4 

h + 



h + ■ ■ ■ 

We use the corresponding notation for finite continued fractions. 
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Several authors @, ||, 0, [L5], [16| have described how to express the generating function 
for (3 ra (132) with respect to various (k > 1) as a continued fraction. The most general of 
these results is the following, which appears explicitly as 0, Theorem 1]. A special case of 
this result appears as |1^, Theorem 1], the result is implicit in [12, Proposition 2.3], and it 



can be proved by modifying slightly the techniques of || Corollary 7] and 0, Theorem 1]. 
Theorem 5.2 (Branden, Claesson, and Steingrimsson /fj, Theorem 1]) For alii > 1, let Xi 
denote an indeterminate. Then we have 



n 



X k 



Err T k (w) _ 1 %i X\X2 X\X 2 Xz k>i 
ii x k -7 _ t _ — _ -~r~ - ■■■ - i _ 

7reS(132) fc>l 

In the same paper, Branden, Claesson, and Steingrimsson define a Catalan continued 
fraction to be a continued fraction of the form 

1 m mi f»2 

where for all % > 0, the expression mi is a monic monomial in a given set of variables. Roughly 
speaking, Branden, Claesson, and Steingrimsson show |4], Theorem 2] that the multivariate 
generating function for 6(132) with respect to a given countable family of statistics may 
be expressed as a Catalan continued fraction if and only if each statistic in the family is a 
(possibly infinite) linear combination of the r^s and each appears in only finitely many of 
these linear combinations. 

In this section we prove analogues of these results for permutations which avoid 1243 
and 2143. We begin by adapting Krattenthaler's approach in ||, using our bijection ip and 
a result of Flajolet to express the generating function for (3(1243,2143) with respect to rfc, 
k > 1, as a continued fraction. For convenience, we first recall the relevant specialization of 
Flajolet's result. 

Theorem 5.3 (Flajolet g, Th eorem 1]) For all i > 1, let Xi denote an indeterminate. Then 
we have 



n 

X- 



n 



2 

TT Tfe(i") X l X\X2 X\X 2 X^ j=o r x 

> av = 5 — • • . z. (15) 

7?st>i l-xi ~ l-x lX2 - 1- Xl a* 2 x s - - X _-Q x p 

i=0 



Combining with Theorem ^.3| , we obtain the following analogue of Theorem p^2 
Theorem 5.4 For all i > 1, let x« denote an indeterminate. Then we have 



e n 



n 



2 

x T k( n ) — 1 i Xl X\X 2 X\X 2 Xj, i = o 



"* h 1 -i 1 9 * * * n 

1 — xi — 1 — X\X 2 — 1 — Xixix-i — 

ttS 6(1243,2143) k>\ 1 1 Z 1 2 a 



i-n 



i=0 



(16) 
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Proof. This is immediate from Theorem [5.3| , in view of Propositions |4.7| and |4.8| . □ 

Using fll6|) , we can express the generating function for |(3 n (1243, 2143, 12 . . .fc)| as a 
(finite) continued fraction. 
Corollary 5.5 For all k > 1 we /iave 

oo 

V|6 n (1243,2143,12...A;)|x n = l + -^- ... . (17) 

^— ' 1 — x— 1 — x— — 1 — x 

n=0 v y ' 

fc— 1 terms 

Proof. In ([16|) set xi = x, x 2 = X3 = . . . = Xk-i = 1 and x,, = for all i > k. □ 



Curiously, as we prove in Proposition 6.10, the continued fraction on the right side of 



(|T7|) is also equal to the generating function for |(5 n (1243, 2143, 213 . . . k)\. 

Using (|16|), we can also express the generating function for ©(1243,2143) with respect 
to the total number of increasing subsequences as a continued fraction. 
Corollary 5.6 For any permutation n, let m(7r) denote the number of nonempty increasing 
subsequences in it. Then 



_ -| 1 x 1 X Q 2 X( l 4 X( l 2n 

1 — xq — 1 — xq 2 — 1 — xq 4 — — 1 — xq 2 " 

7reS(1243,2143) 111 1 



q m{w) x H = 1 + 



Proof. In ([T6|), set X\ = xq and X{ = q for all i > 2 and use the fact that m = T k- 1=1 

k>l 

By specializing the x^s in fllTf ) in various ways, one can obtain continued fractions of 
many other interesting forms. As an example, we have the following result. 
Corollary 5.7 For any permutation n, let m(7i) denote the length of it plus the number of 
noninversions in n. Then 



-II q ^ ^ ^ 

- — q — 1 — q 2 — 1 — o 3 — 1 — o 4 

tt6 6(1243,2143) 



J] g m « = 1 



Proof. In ([Tq) , set xi = X2 = g and Xj = 1 for all i > 3 and use the fact that m = t\ + r^. □ 

We now turn our attention to the question of which statistics on 6(1243,2143) have 
generating functions which can be expressed as continued fractions like the one in ([16]). We 
begin by specifying which continued fractions we wish to consider. 

By a Schroder continued fraction we mean a continued fraction of the form 

1 + m ° mi 1112 

1 — niQ — 1 — mi — 1 — itl 2 — 

in which rrii is a monic monomial in a given set of variables for all i > 0. Observe that if 
fi, f2, hi ■ ■ ■ are (possibly infinite) linear combinations of the r^s with the property that each 
Tfc appears in only finitely many /j, then by specializing the x^s appropriately in ( |i7j| ) we can 
express the generating function 



J2 x w TT 



7rS6(1243,2143) fe>l 
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as a Schroder continued fraction. For instance, when only fi is present, we have the following 
corollary of Theorem |6.5| . 

Corollary 5.8 Let Ai, A2, • • • denote nonnegative integers and let f denote the statistic 



f = ^ A fc r fc 



k>l 



on 6(1243,2143). Then 
q fM x [ 



vreS(1243,2143) 
1 + 



xq f(l) x „/(12)-/(l) x /(123)-/(12) x /(1234)-/(123) 



1-XqfW - 1 - Xg/( 12 )-/« - 1 - X g/(123)-/(12) _ J _ xg /(1234)-/(123) 

Proof. In (|T6|) set = xq Xl and = q Xi for alii > 2 to obtain 



= 1 + y 



xq Xl xq Xl ^^ 2 /£g-^i+2A2+A3 



xo Al — 1 — xo Al+A2 — 1 — xq ,Al+2A2+A3 — 

7ree(1243,2143) 11 -i 

fc-1 

Now the result follows from the fact that /(123 . . . k) - /(12 ...k — l) = J2 (^T 1 ) A, for all 

i=0 ' 

k > 2. □ 

Modifying the proof of j|, Theorem 2] slightly, we now show that linear combinations of 
the TfcS are the only statistics on (3(1243, 2143) whose generating functions can be expressed 
as Schroder continued fractions. In order to do this, we first set some notation. 

Let A denote the ring of matrices with integer entries whose rows and columns are indexed 
by the positive integers, and which have the property that in any row only finitely many 
entries are nonzero. Then each element of A corresponds to an infinite family of statistics 
on 6(1243,2143). 

Definition 5.9 For all A e A and all n > 1, let r^ n denote the linear combination of the 
TkS whose coefficients appear in the nth column of A. That is, 

i~A,n = 22 AknTk ( 18 ) 

k>l 

for all A G A and all n > 1 . 

Since each element of A corresponds to a family of statistics on 6(1243,2143), each 
element also has an associated multivariate generating function. 

Definition 5.10 For all i > 1, let qi denote an indeterminate. For all A £ A, we write 
Fa(c{) to denote the generating function given by 

fA(q)= E IK^ ( 19 ) 

7ree(1243,2143) k>l 
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We also associate with each element of A a Schroder continued fraction. 
Definition 5.11 For all i > 1, let qi denote an indeterminate. For all A £ A, we write 
C A (q) to denote the continued fraction given by 



cu(q) = i 



n «r 

k>l 



k>l 



k>l 



Uik 

k>l 



i - n C k - i - n it 



A,, 



(20) 



We now give our analogue of |4], Theorem 2]. Although the proof is nearly identical to 
the proof of |4], Theorem 2], we include it here for completeness. 

Theorem 5.12 Let B denote the matrix in A which satisfies Bij = Ql-J ■ (Here we use the 
convention that Q) = whenever i < j.) Then for all A 6 A, 



and 



F A (q) = C BA (q) 



CU(q) = F B -i A (q). 



(21) 



(22) 



In particular, the set of Schroder continued fractions is exactly the set of generating functions 
for countable families of statistics on 6(1243,2143) in which each statistic is a (possibly 
infinite) linear combination of the r k s and each r k appears in only finitely many statistics. 



Proof. To prove (|2l|) , we apply (|16|) with Xi 



Y[ Qj !J for all z > 1. We have 



1 + 



Xi 



XiX 2 



2 

X1X2X3 



1 — X\ — 1 — X\X 2 — 1 — ^1X2X3 — 



e n 



x 



(by H 



7re 6(1243,2143) k>l 



e n iK" 

7ree(1243,2143) fc>l \j>l 

e n«r 

7ree(1243,2143) j>l 



(by (|TB|) and (0)), 
(-l) i+i (py so fi- 1 G ^. Therefore 



we 



as desired. To prove (|22D , observe that (B 1 )y 
may replace A with in (HD to obtain fl2|). □ 

Remark There is a result for finite Schroder continued fractions which is analogous to 
Theorem |5.12| . Specifically, fix k > 2 and let Ak denote the ring of k — 1 by k — 1 matrices 
with integer entries. If we replace A with A k and 6(1243, 2143) with 6(1243, 2143, 12 ... k) 
in Definitions [5]9] and |5.9| then the resulting analogue of Theorem |5.12| follows by an argument 
almost identical to the proof of Theorem [5. 12 
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6 Generating Functions Involving Chebyshev Polyno- 
mials 



In this section we will be interested in connections between pattern-avoiding permutations 
and Chebyshev polynomials of the second kind. We begin by recalling these polynomials. 

Definition 6.1 For all n > —1, we write U n (x) to denote the nth Chebyshev polynomial 

sin((n + l)t) 

of the second kind, which is defined by LLi(x) = and U n (cost) = - — for n > 0. 

These polynomials satisfy 

U n (x) = 2xU n - 1 (x) - U n - 2 (x) (n > 1). 



shit 



(23) 



We will find it useful to reformulate the recurrence in (|23|) as follows. 
Lemma 6.2 For all n > 1, 



1 — x 



1 — X ( 1 — X 



X 



1 — X 



(24) 



Proof. This is immediate from fl2"3"|). □ 

Chebyshev polynomials of the second kind have been found to have close connections 
with 6(132); see, for instance, [|, [| [12], 13, 15]. Two such connections are given in the 
following results. 

Theorem 6.3 (Krattenthaler Theorem 2], Mansour and Vainshtein \12\ , Theorem 3.1], 
Chow and West /Q, Theorem 3.6, second case]) For all k > 1 we have 



£)|6 ft (132,12...fc)| 



x' 



n=0 



U ^-l (2^ 



Theorem 6.4 (Krattenthaler JQ, Theorem 3], Mansour and Vainshtein \12\ , Theorems 3.1 
and 4-1]) Let r > 1, b > 1, and k > 2 satisfy ( fe+ ^ _1 ) < r < ( t )■ Then the generating 
function for the number of 132-avoiding permutations which contain exactly r subsequences 
of type 12 ... k is given by 



En 



i=2 



h-i + h ~ 1 



1 (57s)) 

h+1 



i(Ji-l)+E h 



x 



3=2 



«*)) 

where the sum on the left is over all sequences li, I2, ■ ■ ■ , h of nonnegative integers such that 

'k + i-2^ 



i=l 



r. 



In this section we prove analogues of Theorems |6.3j and |6.4| for (5 n (1243, 2143). We begin 



with an analogue of Theorem |T3 . 
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Theorem 6.5 For all k > 1 we have 



^|6 n (1243,2143,12...A;)|x n = 1 + V Y 7 



n=0 



(25) 



and 



V|6 n (1243, 2143, 213... k)\x n = 1 + / \ • 



n=0 



1 >Gs) 



(26) 



Before pressing on to the proof of Theorem xo, we observe that when we set A; = 3 in 
']) and (p6|) we find that 

CO .. 

^|6 n (2143,123)|x n = l + x: 



n=0 

and 



1 — 3x + x 2 



|e„(1243,213)|x™ = 1 + x- 



1 — 3x + x 2 

n=0 

These generating functions were originally found by West 0, Example 9 and Table 1] , using 
generating trees. 

Our next result is the key to our proof of Theorem |6.5| . To state it, we first set some 
notation. 

Definition 6.6 For any set R of permutations, let Rn denote the set of permutations ob- 
tained by appending the smallest possible positive integer to each permutation in R. 

Example 6.7 If R = {42531,4231,312} then Rn = {425316,42315,3124}. 

Theorem 6.8 Let R denote a nonempty set of permutations and set 

oo 

P(x) = |6n(1243,2143,/2)|x n (27) 

?1=0 

and 

oo 

Q(x) = |6„(1243, 2143,^)1^. (28) 

n=0 

Then 

Proof. Arguing as in the proof of Proposition |3.3j (i), we find that for all n > the map 

6 n (1243, 2143, Rn) — > 6 n+1 (1243, 2143, Rn) 

71 I — > U + 1 , 7T 
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is a bijection between (5 n (1243, 2143, Rn) and the set of all permutations in (5 n +i(1243, 2143, Rn) 
which begin with n + 1. Arguing as in the proof of Proposition |3]6](i), we find that for all i 
and all n such that 1 < i < n — 1, the map 



6,(1243, 2143, x 6 n _; (1243, 2143, #n) 

(7Tl,7r 2 ) 



6 n (1243,2143,i?n) 

7Tl * 7T2 



is a bijection between 6^(1243, 2143, R) x 6 n _j(1243, 2143, and the set of permutations 
in 6 n (1243, 2143, Rn) for which n(i + 1) = n. Combining these two bijections, we find that 

Q{x) = 1 + xQ{x) + (P(x) - l){Q{x) - 1). 

Solving this last equation for Q(x), we obtain (|29|) . □ 

As we show next, if P(x) is a rational function then Q(x) has a simple form. 
Corollary 6.9 Let R denote a nonempty set of permutations. If 

°° f( x ) 

22 |©n(1243,2143,i?)|x n = 1 + x J ' 

71=0 



g{x) 



then 



Proof. By 



^|6 n (1243,2143,i?n)|x n = 1 + x— 



g(x) 



n=0 

we have 



(1 - x)g(x) - xf(x) 



^|6 n (1243,2143,i?n)|x n 



1 — x 



g(x) 



n=0 



1 — X + X 



g(x) 



as desired. □ 



1+x 



9{?) 



(1 - x)g(x) - xf{x)' 



Using Corollary |6.9| it is now routine to prove Theorem pl> . 



Proof of Theorem \6. | . We argue by induction on k. It is routine to verify that fl25|) holds 
for k = 1 and k = 2, so we assume (^) holds for k — 2 and k — 1. Then by Corollary 
and (|24]) we have 



5^|6 n (1243,2143,12 . . .k)\ 



x 



1 + 



v^4 , (ft) 



n=0 



1 — X T J I l — X 



^-2 ( ft 



1 + 



y/xUf 



k-2 



1-a 
2^ 



^-1 (ft) 
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as desired. The proof of (p6|) is similar to the proof of (Pq). □ 



We now have a corollary of Theorem |6.5| involving the continued fraction which appears 
in©- 

Corollary 6.10 For all k > 2 we have 



|S„(1243, 2143, 2134 . . . k)\x r ' 



1 + 



x 



x 



X 



n=0 



X 



— X — 

v 

h— 1 terms 



X 



Proof. This is immediate from Theorem |6.5| and Corollary |5.5| . □ 

Next we turn our attention to our analogue of Theorem |6.4j . 
Theorem 6.11 Fix r > 1, k > 2, and b > such that ( k t b ) < r < ( k+ l +1 ) ■ Then we have 



E^zn 



8=0 



h + l i+ x + m* - 1\ (l i+x + mA \ u k-2 \ 



l i+ i + mi 



rrii 



lo-l 



Iq+1 



(30) 

Here the sum on the left is over all permutations in (3(1243,2143) which contain exactly r 
subsequences of type 12 ... k. The sum on the right is over all sequences IqJi, ■ ■ ■ , h, an d 
mo, mi, . . . , nib of nonnegative integers such that 



(31) 



Throughout we adopt the convention that (^) = 1 and (^J = for any integer a. 

To prove this theorem, we first need to set some notation and prove some preliminary 
results. We begin with a certain matrix. 

Definition 6.12 For all k > 0, we write to denote the k + 1 by k + 1 tridiagonal matrix 
given by 



( x v/i 

y/x X yfx 
\fx X y/x 






V o 



























\ 





X 



X 



X 








X X 







x 





x 

x ) 



The matrix A^ is closely related to generating functions for various sets of Schroder paths. 
To describe this relationship, we let S (r, s) denote the set of lattice paths involving only east 
(1,0), north (0,1), and diagonal (1,1) steps which begin at a point at height r, end at a 
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point at height s, and do not cross the lines y — x = k and y = x. For any such path ir 
which begins at (x\, y\) and ends at (x 2 , y 2 ), we write 1(tt) = ~ (x 2 + y 2 — %\ — Hi)- The next 
lemma summarizes the relationship between A k and the generating function for S (r, s) with 
respect to /. 

Lemma 6.13 For all k > and all r and s such that < r, s < k we have 



7r£<S(r,s) 



X 



;-ry+'det(/-4 fe ;s,r) 
det(/-40 



i/ere J is the identity matrix of the appropriate size and det(7 — A k \ s,r) is the minor of 
I — Ak in which the sth row and rth column have been deleted. 

For the sake of brevity we omit the proof of Lemma |6.13| . An outline of this proof can 
be obtained by modifying slightly the outline of the proof of || Theorem A2]. 

The matrix A k is also closely connected with Chebyshev polynomials of the second kind. 
In particular, we have the following result. 

Lemma 6.14 For all k > 0, 



k+l 



1 — X 



det(J - A k 



Proof. We argue by induction on k. It is routine to verify that the result holds for k = 
and k = 1, so we assume the result holds for k — 2 and k — 1. By expanding det(J — A*.) along 
the bottom row, one can show that (V^T (fe+1) det(I - A k ) satisfies By our induction 

assumption we find the result holds for k, as desired. □ 



As a warm-up for our proof of Theorem |6.11| , we use Lemmas |6.13| and |6.14| to give 
another proof of (|25|) . 



Another proof of (WW- First observe that if F(x) is the generating function on the left side 
of (fffi) and G(x) is the generating function for Schroder paths from (0, 0) to (n, n) which do 
not exceed height &; — 2, then in view of (|14 ) we have F(x) = l + xG(x) 

oTTB| and g we find that G(x) = * 2 ^ 2 /*\ ■ Now follows. □ 



Combining Lemmas 



l-x \ 



We now prove Theorem |6.11 



Proof of Theorem \6.l\ First observe that if F[x) is the generating function on the left 
side of (^) and G(x) is the generating function for the set of Schroder paths from (0, 0) to 
(n, n) for which Tfc(7r) = r, then in view of (13) we have F(x) = xG(x). With this in mind, 
we find G(x). 

To compute G(x), we observe that any Schroder path ir with Tfc(7r) = r can be constructed 
by the following procedure in exactly one way. 

1. Choose lo,h, ■ ■ ■ and mo, mi, . . . , nib such that ( ffTp holds. Construct a sequence of 
east and diagonal steps which contains exactly U east steps at height k + i — 1 and rrii 
diagonal steps at height k + i — 1 for < i < b and which satisfies both of the following. 
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(a) The step immediately preceeding a step at height j is either an east step at height 
j + 1 or less, or a diagonal step at height j or less. 

(b) All steps after the last east step at height j are at height j — 1 or less. 

2. After each east step at height k — 1 except the last, insert an (possibly empty) upside- 
down Schroder path of height at most k — 2. 

3. Before the first step insert a path from height to height k — 2 which does not exceed 
height k — 2. 

4. After the last step insert a path from height k — 2 to height which does not exceed 
height k — 2. 

Since the choice at each step is independent of the choices at the other steps, and since 
every sequence of choices results in a path of the type desired, G(x) is the product of the 
generating functions for each step. 

To compute the generating function for step 1, suppose we have fixed Iq, l±, . . . , 4, and 

b 

J2 (h+rrii) 

mo, nil, ■ ■ ■ , Tnb', then each of the resulting partial paths will have generating function x j=0 

To count these paths, we construct them from the top down. That is, we first arrange the 

nib diagonal steps at height b; there is one way to do this. We then place the lb east steps at 

height b so that one of these steps occurs after all of the diagonal steps. There are 

ways to do this. We then place the nib-i diagonal steps at height b — 1 so that none of these 

steps immediately follows a diagonal step at height b. There are ^ 6+mt '- 1 ) ways to do this. 

Proceeding in this fashion, we find that the generating function for step 1 is 



where the sum is over all sequences lo,h, ■ ■ ■ ,h, and m , mi, . . . , m b of nonnegative integers 
which satisfy (pl|). Using Lemmas |6.13| and |6.14| , we find that the generating function for 
step 2 is equal to 



' - 2 (fcl) 



and the generating functions for steps 3 and 4 are both equal to 

(-l) fc - 2 

v^4-i (fef ) ' 



(33) 



(34) 



Taking the product of x, the quantities in (j32D and (p3|) , and the square of the quantity in 
(|34|), we obtain (|30|), as desired. □ 



The following special case of Theorem 3.11 is of particular interest. 
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Corollary 6.15 For all k > 2 we have 

x 



y> H = — - — 2' < 35 > 
(^-. (&)) 

i/ere t/ie sum on the left is over all permutations in 6(1243, 2143) which contain exactly one 
subsequence of type 12 ... k. 

Proof. Set r = 1 in Theorem |6.11| . d 



7 Permutations in 6(1243,2143) Which Avoid Another 
Pattern 



In Section^ we used Theorem |6.8| to obtain generating functions for 6 n (1243, 2143, <x34 . . . k), 



where a £ 62. In this section we use Theorem |6.8| to obtain generating functions for 
6 n (1243, 2134, a) for other interesting choices of a; in some cases we give explicit enumera- 
tions. We begin with a = 231. 

Proposition 7.1 For all n > 2, 



Moreover, 



|S„(1243, 2143, 231)| = (n + 2)2 n ~ 3 . (36) 



^|6 w (1243,2143,231)|x n = l + x 1 _ ; (37) 

n=0 l > 



Proof. We first prove (071). Let S(x) denote the generating function on the left side of 
(|3). Observe that if tt e 6 n (1243, 2143, 231) then 7r _1 (n) = 1, ^(n) = 2, or 7r _1 (n) = n. 
Therefore we may partition ©(1243,2143,321) into the following four sets: 

Ti = {it e 6(1243, 2143, 231) | \n\ < 3}; 

T 2 = {n e 6(1243, 2143, 231) | |tt| > 4 and tt(1) = |tt|}; 

T 3 = {% e 6(1243,2143,231) | |tt| > 4 and tt(2) = |tt|}; 

T 4 = {tx E 6(1243,2143,231) | |tt| > 4 and 7r(|7r|) = |tt|}. 

The generating function for T\ is l+x+2x 2 +5x 3 . The map from 6 n „i(1243, 2143, 231) which 
carries it to n, tt is a bijection between 6 n _i(1243, 2143, 231) and the set of permutations in 
T2 which have length n, so the generating function for T2 is x(S(x) — 1 — x — x 2 ). Now observe 
that if n > 4 and n E 6 n (1243, 2143, 231) has tt(2) = nthenTr(l) = 1. It follows that the map 
from 6 n _2(132, 231) to 6 n (1243, 2143, 231) which carries 7r to 1, n, tt, where % is the sequence 
obtained from ir by adding one to every entry, is a bijection between 6 n _2(132, 231) and the 
set of permutations in T 3 which have length n. By fTTj , Proposition 9], the generating function 

for 6 n (132, 231) is 1 H , so the generating function for T 3 is x 2 I 1 H 1 — x I . 

1 — 2x \ 1 — 2x I 
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Finally, the map from 6 n _i(1243, 2143, 231) to 6 n (1243, 2143, 231) which carries tt to %,n 
is a bijection between © n _ 1 (1243, 2143, 231) and the set of permutations in T 4 which have 
length n. Therefore the generating function for T4 is x(S(x) — 1 — x — 2x 2 ). Combine these 
observations to find that 

S( x ) = l+x+2x 2 +5x 3 +x(S(x)-l-x-2x 2 )+x 2 ( 1 + 1 - x J +x{S{x)-l-x-2x 2 ). 

Solve this equation for S(x) to obtain (p7|). 
Line (|36|) is immediate from (|3"7|). □ 

Combining Proposition [Z7T] with Theorem and arguing as in the proof of Theorem 
|6.5| , we find the generating function for (3 n (1243, 2143, 2314 . . . k) for any k > 4. 

Proposition 7.2 Sei r2(x) = (x — l) 2 and 
r fc (x) = (1 - 2x) 2 (v^) fc - 3 f/ fc - 3 - (1 - x) 2 {^ 2 U k ., (k > 3). 



Then for all k > 3 we /iave 

r fc _i(x) 
r fc (x) 



^|6 n (1243, 2143, 2314... k)\x n = 1 + 



x- 



n=0 



Next we consider a = 321. We begin with a lemma concerning which products 7i\ * 1x2 
avoid 321. 

Lemma 7.3 Fix i and n such that 1 < i < n — 2, fix %i G (5j(1243, 2143, 321) ; and fix 
tt 2 G 6 n _i(1243, 2143, 321). T/ien tti * tt 2 G 6„(1243, 2143, 321) if and only if all of the 
following hold. 

(i) At least one of i\\ and t\2 begins with 1. 

(ii) The entries 2,3, ... ,i are in increasing order in n\. 

(iii) The entries 7^(2), ^(3), . . . , 712 (n — i) are in increasing order in 7r 2 . 



Proof. {=>) Suppose 7Ti * 7r 2 G 6„(1243, 2143, 321). If (i) does not hold then the entries 
(7T1 * 7r 2 )(l), n — i, 1 form a subsequence of type 321 in 7Ti * 7r 2 . If (ii) does not hold then the 
smallest entry to the right of n, together with two of the entries among 2,3, ... ,i which are 
in decreasing order, form a subsequence of type 321 in tt\ * 7r 2 . If (iii) does not hold then the 
n, together with two of the entries among 712(2), 7^(3), ... ,7^(71 — 1) which are in decreasing 
order, form a subsequence of type 321 in iii * tv 2 - Since we have a contradiction in each case, 
we find (i),(ii), and (iii) hold. 

(<^=) By Proposition |3]6|(i), the permutation 7i"i * 7r 2 avoids 1243 and 2143. Now observe 
that by (i)-(iii), one of the following holds. 

1. There exists k, 1 < k < i — 1, such that 

m * tt 2 = n — i + l,n — i + 2, . . . ,n — i + k,l,n — i + k + 1, . . . ,n — l,n,2,3,...,n — i. 
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2. There exists k, 1 < k < n — i, such that 

7Ti * tt 2 = kj n — % + l,n — i + 2, . . . , n — 1, n, 1, 2, . . . , k — 1, k + 1, . . . , n — i. 

It is routine to verify that in either case, 7Ti*7r 2 avoids 321. Therefore 7ri*7r 2 G (3 n (1243, 2143, 321), 
as desired. □ 

We can now enumerate 6 n (1243, 2143, 321). 
Proposition 7.4 For all n > 1 we have 

|6 n (1243, 2143, 321)|= f n ~ ^ + (™ ~ ^ +2^" *J + 2 ( W g J- ( 38 ) 



Moreover, 



OO 



e„(1243, 2143, 321) |x n = 1 + x^— ^L^L. (39) 



' ' ' ■ - (1-x) 4 

n=0 v 7 



Proof. We first prove (|39|). Let S'(x) denote the generating function on the left side of (|39|) . 
By Lemma |7.3| , we may partition (5(1243, 2143, 321) into the following four sets: 

7i = {tt G 6(1243, 2143, 321) | tt = or tt(1) = n}; 

T 2 = {tt G 6(1243,2143,321) | |vr| > 2 and 7r(|yr|) = |tt|}; 
T 3 = {nE 6(1243, 2143, 321) | tt = tti * 7r 2 , |tt 2 | > 2, 7^(1) = 1}; 
T A = {tt G 6(1243, 2143, 321) | tt = tti * 7r 2 , |tt 2 | > 2, 7r x (l) ^ 1, tt 2 (1) = 1}. 
Observe that 71 consists of exactly those permutations of the form n, 1, 2, 3 . . . , n — 1, so 

the generating function for T\ is . Observe that the map from 6 n _i(1243, 2143, 321) 

1 — x 

to 6 n (1243, 2143, 321) which carries vr to n,n is a bijection from 6 n _i(1243, 2143, 321) to 
the set of permutations in T 2 which have length n, so the generating function for T 2 is 
^(^(x) — 1). Observe that if 7r = ttx * tt 2 and 7r G T 3 then 7Ti has the form 1, 2, . . . , |*7Ti| and 
7r 2 has the form fc, 1, 2, . . . , k — 1, A; + 1, . . . , |7r 2 | for some k, 1 < k < |tt 2 |. It follows that the 

generating function for T 3 is — J ^yj ^ — • Finally, observe that if n = 71*1 * 7r 2 

and 7r G T4 then 7r 2 has the form 1,2,..., |7T 2 | and there exists k, 2 < k < \tti\, such that 
7Ti has the form 2, 3, . . . , k, 1, k + 1, . . . , |7i"i|. It follows that the generating function for T4 is 

. Combine these observations to find that 



X^ "\ f x^ 



'l-xf 



1 2 2 4 
S(x) = + X(S(X) - 1) + - + 



x ' ' ' (1 — x) 3 1 — x (1 — a^ 3 



Solve this equation for S(x) to obtain (|3~9l). 
Line (|38|) is immediate from (^). □ 

Combining Proposition TTQ with Theorem |6]3 and arguing as in the proof of Theorem 



BTq , we find the generating function for 6 n (1243, 2143, 3214 . . . k) for any k > 4. 



27 



Proposition 7.5 For all k > 2, set 

r k (x) = -(1 + 2x - x 2 ){V^) k+2 U k + ~ ^ + 2x 2 + l)^" 1 ^-! 

Then for all k > 3 we have 



J^|6 n (1243, 2143, 3214... k)\x n = 1 + 



n=0 



r k (x) 



At the end of Section |3| we remarked that the results of that section can be generalized 
to &(T k ), where T k is the set of all permutations in & k which end with k, k — 1. Our 
results in this section build almost exclusively on the results of Section fj[ so they can also 
be generalized to &(T k ). 



8 Permutations in ©(1243, 2143) Which Contain Another 
Pattern 

In Section ^ we used Theorem |6.8| to obtain generating functions for 6 n (1243, 2143, a) for 
various a; these results are elaborations on Theorem |6.5| . In this section we prove an analogue 
of Theorem p.8| for permutations in (5 n (1243, 2143) which contain a given pattern a exactly 
once. We then use this result to prove several results which elaborate on Theorem |6. 1 1| . We 
begin by setting some notation. 

Definition 8.1 Fix a nonempty permutation r G 6(1243,2143) and set k — |r| + 1. We 

write G T (x) to denote the generating function for those permutations in (5(1243, 2143) which 
contain exactly one subsequence of type r. We write H T (x) to denote the generating function 
for those permutations in 6(1243, 2143) which avoid t. We write J T (x) to denote the gener- 
ating function for those permutations in 6(1243, 2143) which avoid r, k and contain exactly 
one subsequence of type r. 

Our analogue of Theorem |6l| is the following result, which gives G Tjk (x) in terms of J T (x) 
and H T (x). 

Theorem 8.2 Fix a nonempty permutation t e 6(1243,2143) and set k = |r| + 1. Then 

Proof. Observe that we may partition the set of permutations in 6(1243,2143) which 
contain exactly one subsequence of type r, k into the following three subsets. 

Ti = {tt I 7i begins with |7r|}; 

T2 = {tt I tt = Tii * 7r 2 ; 71"! contains exactly one r and no r, k; tt 2 avoids r, k}; 
T3 = {n I 7T = 7Ti * 7T2 ; Ti\ avoids r; 712 contains exactly one r, A;}. 
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The generating function for T\ is xG Tt k(x), the generating function for T 2 is J T (x)(H T ^(x) — 1), 
and the generating function for T 3 is G Ti k{x)(H T (x) — 1). Combine these observations to find 
that 

G T:k (x) = xG Tik (x) + J T (x)(H T! k(x) - 1) + G T;k (x)(H T (x) - 1). 

Solve this last equation for G Ty k{x) and use Theorem |6l^ to eliminate H T ^(x). Simplify the 
result to obtain (HU1). □ 



Theorem |8.2j is particularly useful when J T (x) = G T (x); next we describe for which r this 
occurs. 

Proposition 8.3 Fix a permutation r e (5(1243,2143) such that |r| > 1. Then J T (x) = 
G T (x) if and only if the last entry of t is |r|. 

Proof. (=>) Suppose J T (x) = G T (x) and observe that this implies that if ir contains exactly 
one subsequence of type r then n avoids r, \t\ + 1. However, if the last entry of r is not |r| 
then r, |r| + 1 itself contains exactly one subsequence of type r and does not avoid r, \t\ + 1. 
This contradiction implies that the last entry of r is |r|. 

(<^=) Suppose the last entry of r is \r\. If n contains r and r, \r \ + 1 then 7r contains at 
least two subsequences of type r. Therefore if n contains exactly one subsequence of type r 
then it avoids r, |r| + 1; it follows that J T (x) = G T (x). □ 

Using Theorem |8.2j and Proposition |S.3|, one can compute G a (x) for several infinite fam- 
ilies of permutations. Nevertheless, we content ourselves here with computing G\2...k(x) and 
G213.../C (x). We begin by computing Gi2...k(x), which amounts to giving another proof of 
Corollary |6.15| . 



Another Proof of Corollary \6.15] We argue by induction on k. To prove the result for k = 2, 



set t = 1 in (f40P and observe that J\{x) = x and Hi(x) = 1. Now suppose the result holds 
for k - 1. Using (gg), Proposition and (f£§) we find 

xG 12 ... k -i{x) 



G 12 ...k(x) 



x 



X 



(u k -i 



as desired. □ 



We now compute G 2 i 3 ...fc(a;). 
Proposition 8.4 For all k > 3 we have 



G 21 ,.,(*) = •- r l i i ';:t (4i) 
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Proof. First observe that the only permutations in (5(1243, 2143) which contain exactly one 
subsequence of type 21 and do not contain a subsequence of type 213 are 21 and 132, so 
J21 (x) = x 2 + x 3 . Now the result follows from (|40"D , Proposition |3.3| , and ( f24"l) by induction 
on k. □ 

We conclude this section by showing that in a certain sense, there is no analogue of <p 
which behaves as nicely with respect to 213 ... k as ip does with respect to 12 . . . k. 

Corollary 8.5 There is no bijection uj : S n — > © n+ i(1243, 2143) such that for every n G 
S n , the quantity t^k) is equal to the number of subsequences of type 213 . . .k in u(ir). 

Proof. If such a bijection existed then we would have Gi2...k{x) = G213. ..&(£) for all k > 3, 
which contradicts (|35|) and (Ed]). □ 



We remark that one can also prove Corollary |8.5| by observing that if such a bijection 
existed, then there would be the same number of permutations in ©4(1243,2143) which 
contain exactly one subsequence of type 123 as there are containing exactly one subsequence 
of type 213. However, there are 6 of the former (2314, 1423, 2341, 1342, 4123, 3124) and 
only 5 of the latter (3241, 2413, 1324, 3142, 4213). As this example illustrates, subsequences 
of type 213 ... A; are more plentiful in (5 n (1243, 2143) than subsequences of type 12 ... k. 



9 Open Problems 

We conclude with some open problems suggested by the results in this paper. 

1. In view of (p5|), ( p6| ) and Corollary p~9| , there exists a bijection p : S n — > (5 n+ i(1243, 2143) 
such that for all k > 2, the Schroder path n has height k — 2 or less if and only if 
p(tt) avoids 213 ... A;. Give a combinatorial description of such a bijection. Then give 

a combinatorial description of the statistic a^, where 0^(71") = Tfc(p _1 (7r)) for all tc G 
© n (1243,2143). Alternatively, give a bijection u : 6 n (1243, 2143) — > 6 n (1243, 2143) 
such that for all k > and any permutation ir G (5 n (1243, 2143), we have that n avoids 
12 ... k if and only if u}(n) avoids 213 . . . k. 

2. For any permutation 7r G (5 n (1243, 2143) let T n (x) denote the generating function given 
by 

00 

%(x) = ^|6 n (1243,2143,7r)|x n . 

n=0 

Then the case of Theorem |3.8| in which \R\ = 1 amounts to a recurrence relation giving 
T^i{x) in terms of T n (x). Find similar relations for T 7T1 ^ 7T2 (x) in terms of T ni (x) and 



7^ 2 (x) and for T n7T (x) in terms of T^x). Alternatively, find an analogue of [14], Theorem 
2.1] for 6(1243,2143). 

3. Give a combinatorial proof of the fact that 

|6„(1243, 2143, 231) | = (n + 2)2 n ~ 3 (n > 2). 
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4. Give a combinatorial proof of the fact that 

|6 n (1243,2143,321)|= f"" 1 ! + (^ 1 )+2^ 1 ) + 2 ("~3 ^ (n " ^ 

5. For k > 3 and r > 2, find the generating function for the number of permutations in 
(5 n (1243, 2143) which contain exactly r subsequences of type 213 ... A;. 
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